Medical image analyzing system and method thereof

ABSTRACT

A medical image analyzing system and a medical image analyzing method are provided and include inputting at least one patient image into a first model of a first neural network module to obtain a result having determined positions and ranges of an organ and a tumor of the patient image; inputting the result into a plurality of second models of a second neural network module, respectively, to obtain a plurality of prediction values corresponding to each of the plurality of second models and a model number predicting having cancer in the plurality of prediction values; and outputting a determined result based on the model number predicting having cancer and a number threshold value. Further, processes between the first model and the second models can be automated, thereby improving identification rate of pancreatic cancer.

BACKGROUND 1. Technical Field

The present disclosure relates to image analyzing system and method, andmore particularly, to medical image analyzing system and method.

2. Description of Related Art

In the current medical practice, pancreatic cancer is one of the cancersthat are difficult to detect early, and the survival rate of pancreaticcancer decreases significantly once the tumor size exceeds 2 cm. In theprior art, computerized tomography (CT) imaging is currently the mainmethod for detecting and evaluating pancreatic cancer, but the detectionefficiency still depends on the personal experience of radiologists. Forexample, when the tumor size is less than 2 cm, about 40% of the tumorcannot be detected, which reflects the fact that manual review anddiagnosis are too subjective, and it is easy to misdiagnose due to humanfactors.

Therefore, there is a need in the art to propose a medical imageanalyzing system and a medical image analyzing method for identifyingpancreatic cancer and improving the identification rate of pancreaticcancer.

SUMMARY

In view of the aforementioned problems of the prior art, the presentdisclosure provides a medical image analyzing system, which comprises afirst neural network module having a first model and configured to inputat least one patient image into the first model to obtain a resulthaving determined positions and ranges of an organ and a tumor of thepatient image; a second neural network module having a plurality ofsecond models and configured to respectively input the result having thedetermined positions and ranges of the organ and the tumor of thepatient image into the plurality of second models to obtain a pluralityof prediction values corresponding to each of the plurality of secondmodels and a model number predicting having cancer in the plurality ofprediction values; and a determining module configured to output adetermined result based on the model number predicting having cancer anda number threshold value.

In the aforementioned medical image analyzing system, the presentdisclosure further comprises a database stored with a plurality ofimages, organ position and range markers and tumor position and rangemarkers, wherein the plurality of images, the organ position and rangemarkers and the tumor position and range markers are interlinked to useas a first training set.

In the aforementioned medical image analyzing system, the first neuralnetwork module is trained to obtain the first model based on the firsttraining set, wherein the first neural network module is a modelsearched by using a coarse-to-fine neural structure search (C2FNAS), andwherein the first neural network module uses an Adam optimizer and acosine annealing learning rate scheduler to adjust a learning rate in arange of 10⁻⁴ to 10⁻⁵, and a loss function is set to a Dice losscombined with a categorical cross-entropy loss.

In the aforementioned medical image analyzing system, the first neuralnetwork module obtains a result having determined positions and rangesof an organ and a tumor of the plurality of images by inputting theplurality of images into the first model and uses the result as a secondtraining set.

In the aforementioned medical image analyzing system, the second neuralnetwork module is trained to obtain the plurality of second models basedon the second training set, wherein the second neural network module isa DenseNet-121, and wherein the second neural network module uses anAdam optimizer and a cosine annealing learning rate scheduler to adjusta learning rate in a range of 10⁻⁴ to 10⁻⁵, and a loss function is setto a binary classification loss.

In the aforementioned medical image analyzing system, the second neuralnetwork module evenly divides the plurality of images and the secondtraining set into a plurality of folds, and repeatedly uses one of theplurality of folds as a validation set and the others of the pluralityof folds as a training set in a non-repetitive manner to train andobtain the plurality of second models.

In the aforementioned medical image analyzing system, the presentdisclosure further comprises a threshold-value selection moduleconfigured to plot a plurality of curves for the plurality of predictionvalues, wherein a plurality of threshold values for determining whetherthere is cancer are determined from each of the plurality of curves,such that the second neural network module determines whether theplurality of prediction values predict having cancer based on theplurality of threshold values.

In the aforementioned medical image analyzing system, the plurality ofcurves are receiver operating characteristic curves, and the pluralityof threshold values are corresponding threshold values corresponding tomaximum values of a plurality of Youden indexes or a combination of asensitivity and a specificity.

In the aforementioned medical image analyzing system, the plurality ofYouden indexes are calculated from the sensitivity and the specificitycorresponding to each point in each of the plurality of curves accordingto a formula Youden index=Sensitivity+Specificity−1, wherein thecombination of the sensitivity and the specificity are calculated fromthe sensitivity and the specificity corresponding to the each point ineach of the plurality of curves according to a formula Combination ofSensitivity and Specificity=Sensitivity×N+Specificity, and wherein the Nis any number.

In the aforementioned medical image analyzing system, the determiningmodule respectively calculates corresponding positive likelihood ratiosfor performance of different model numbers predicting having cancer inthe plurality of second models to determine the number threshold value,and wherein the determining module outputs the determined result ofhaving cancer when the model number predicting having cancer is greaterthan or equal to the number threshold value.

In the aforementioned medical image analyzing system, the determiningmodule selects a least number in the different model numbers predictinghaving cancer corresponding to the positive likelihood ratio greaterthan 1 as the number threshold value.

In the aforementioned medical image analyzing system, the presentdisclosure further comprises an image preprocessing module configured toprocess the patient image by resampling, windowing and normalizationbefore inputting the first model or the plurality of second models.

The present disclosure further provides a medical image analyzingmethod, which comprises the steps of: obtaining at least one patientimage; inputting the patient image into a first model of a first neuralnetwork module to obtain a result having determined positions and rangesof an organ and a tumor of the patient image; inputting the resulthaving the determined positions and ranges of the organ and the tumor ofthe patient image into the plurality of second models of a second neuralnetwork module, respectively, to obtain a plurality of prediction valuescorresponding to each of the plurality of second models and a modelnumber predicting having cancer in the plurality of prediction values;and outputting a determined result by a determining module according tothe model number predicting having cancer and a number threshold value.

In the aforementioned medical image analyzing method, the presentdisclosure further comprises the step of: interlinking a plurality ofimages, organ position and range markers and tumor position and rangemarkers to use as a first training set via a database stored with theplurality of images, the organ position and range markers and the tumorposition and range markers.

In the aforementioned medical image analyzing method, the first neuralnetwork module is trained to obtain the first model based on the firsttraining set, wherein the first neural network module is a modelsearched by using a coarse-to-fine neural structure search (C2FNAS), andwherein the first neural network module uses an Adam optimizer and acosine annealing learning rate scheduler to adjust a learning rate in arange of 10⁻⁴ to 10⁻⁵, and a loss function is set to a Dice losscombined with a categorical cross-entropy loss.

In the aforementioned medical image analyzing method, the first neuralnetwork module obtains a result having determined positions and rangesof an organ and a tumor of the plurality of images by inputting theplurality of images into the first model and uses the result as a secondtraining set.

In the aforementioned medical image analyzing method, the second neuralnetwork module is trained to obtain the plurality of second models basedon the second training set, wherein the second neural network module isa DenseNet-121, and wherein the second neural network module uses anAdam optimizer and a cosine annealing learning rate scheduler to adjusta learning rate in a range of 10⁻⁴ to 10⁻⁵, and a loss function is setto a binary classification loss.

In the aforementioned medical image analyzing method, the second neuralnetwork module evenly divides the plurality of images and the secondtraining set into a plurality of folds, and repeatedly uses one of theplurality of folds as a validation set and the others of the pluralityof folds as a training set in a non-repetitive manner to train andobtain the plurality of second models.

In the aforementioned medical image analyzing method, the presentdisclosure further comprises the step of: plotting a plurality of curvesfor the plurality of prediction values by a threshold-value selectionmodule, wherein a plurality of threshold values for determining whetherthere is cancer are determined from each of the plurality of curves,such that the second neural network module determines whether theplurality of prediction values predict having cancer based on theplurality of threshold values.

In the aforementioned medical image analyzing method, the plurality ofcurves are receiver operating characteristic curves, and the pluralityof threshold values are corresponding threshold values corresponding tomaximum values of a plurality of Youden indexes or a combination of asensitivity and a specificity.

In the aforementioned medical image analyzing method, the plurality ofYouden indexes are calculated from the sensitivity and the specificitycorresponding to each point in each of the plurality of curves accordingto a formula Youden index=Sensitivity+Specificity−1, wherein thecombination of the sensitivity and the specificity are calculated fromthe sensitivity and the specificity corresponding to the each point ineach of the plurality of curves according to a formula Combination ofSensitivity and Specificity=Sensitivity×N+Specificity, and wherein the Nis any number.

In the aforementioned medical image analyzing method, the determiningmodule respectively calculates corresponding positive likelihood ratiosfor performance of different model numbers predicting having cancer inthe plurality of second models to determine the number threshold value,and wherein the determining module outputs the determined result ofhaving cancer when the model number predicting having cancer is greaterthan or equal to the number threshold value.

In the aforementioned medical image analyzing method, the determiningmodule selects a least number in the different model numbers predictinghaving cancer corresponding to the positive likelihood ratio greaterthan 1 as the number threshold value.

In the aforementioned medical image analyzing method, the presentdisclosure further comprises the step of: enabling an imagepreprocessing module to process the patient image by resampling,windowing and normalization before inputting the first model or theplurality of the second models.

In summary, the medical image analyzing system and method according tothe present disclosure have higher sensitivity than radiologists inidentifying pancreatic cancer, which means that the medical imageanalyzing system and method according to the present disclosure caneffectively assist radiologists in reducing their clinical misseddiagnosis rate, especially in the case of tumors less than 2 cm in size.Therefore, the situation that about 40% of the tumors cannot be detectedwhen the tumor is less than 2 cm in size can be effectively improved. Inaddition, the medical image analyzing system and method according to thepresent disclosure are automated processes. After directly inputting theoriginal medical image, the medical image analyzing system and methodaccording to the present disclosure can automatically identify thepotential positions and ranges of pancreas and tumor and automaticallyclassify whether the original medical image contains pancreatic cancer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system architecture diagram of a medical image analyzingsystem according to the present disclosure.

FIG. 2 is a flowchart illustrating a medical image analyzing methodaccording to the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The following illustrative embodiments are provided to illustrate thepresent disclosure, these and other advantages and effects can beapparent to those in the art after reading this specification.

Referring to FIG. 1, a medical image analyzing system 1 according to thepresent disclosure includes a first neural network module 11, a secondneural network module 12, a determining module 13, an imagepreprocessing module 14, a threshold-value selection module 15 and adatabase 16. The medical image analyzing system 1 according to thepresent disclosure can operate on devices such as mobile phones, tabletcomputers, laptops, desktop computers, servers, or cloud servers. Thefirst neural network module 11, the second neural network module 12, thedetermining module 13, the image preprocessing module 14 and thethreshold-value selection module 15 can respectively be code fragments,software, or firmware stored in the database 16 or a storage unit, andcan be executed by a processing unit, and can also be implemented byother hardware or combinations of software and hardware. The processingunit can be a central processing unit (CPU), a microprocessor, agraphics processing unit (GPU), or an application specific integratedcircuit (ASIC). The storage unit can be a combination of any type offixed or removable random-access memory (RAM), read-only memory (ROM),flash memory, hard disk, soft disk, database, or similar elements.However, the present disclosure is not limited thereto.

In an embodiment, the first neural network module 11 has a first model.After inputting at least one patient image into the first model, aresult having determined positions and ranges of an organ and a tumor ofthe patient image can be obtained. The second neural network module 12has a plurality of second models. After inputting the result havingdetermined positions and ranges of the organ and the tumor of thepatient image into the plurality of second models, respectively, aplurality of prediction values corresponding to each of the plurality ofsecond models and a model number predicting having cancer in theplurality of prediction values can be obtained. In an embodiment, thepatient image can be a two-dimensional (2D) CT image or athree-dimensional (3D) CT image, and the present disclosure is notlimited as such. Both the first model and the second models are modelstrained by a neural network. The training stages of the first model andthe second models are described below.

The database 16 can store a plurality of images, organ position andrange markers, and tumor position and range markers, and the pluralityof images, organ position and range markers, and tumor position andrange markers are interlinked (e.g., linked to each other). Theinterlink referred herein is to plot the organ position and rangemarkers, the tumor position and range markers, or both on the image atthe same time, and the organ can be a pancreas. In addition, the imagecan be a 2D CT image or a 3D CT image, the organ position and rangemarkers can be a contour of the pancreas, the tumor position and rangemarkers can be a contour of the tumor portion in the pancreas, and theorgan position and range markers and the tumor position and rangemarkers can be marked by an experienced radiologist according to actualdiagnosis data. In an embodiment, all data of the plurality ofinterlinked images, organ position and range markers and tumor positionand range markers are used as a first training set.

The first neural network module 11 can train and obtain the first modelbased on the first training set. The first neural network module 11 is adeep learning model architecture based on SegNet or U-Net, for instance,a model searched by NVIDIA using a coarse-to-fine neural structuresearch (C2FNAS). During a training process, the first neural networkmodule 11 can use Adam optimizer and cosine annealing learning ratescheduler to adjust a learning rate in a range of 10⁻⁴ to 10⁻⁵, a lossfunction is set to a Dice loss combined with a categorical cross-entropyloss, and a batch size is 8.

In an embodiment, the deep learning model architecture based on SegNetor U-Net can be as follows. In a first level, an image of 96×96×96pixels is inputted; after feature extraction is performed by a stem3×3×3 convolution layer, 32 feature images of 96×96×96 pixels areoutputted; after feature extraction using the stem 3×3×3 convolutionlayer with a stride of 2, 64 feature images of 48×48×48 pixels areoutputted; and then entering to a second level. In the second level,another 64 feature images of 48×48×48 pixels are obtained via a 2D 3×3×1convolution layer with said 64 feature images of 48×48×48 pixels(feature 2-1); after using pseudo 3D (P3D) 3×3×1+1×1×3 convolutionlayers, the feature images are added with the feature 2-1 to obtainanother 64 feature images of 48×48×48 pixels (feature 2-2); afterfeature extraction using a 3D 3×3×3 convolution layer with a stride of2, 128 feature images of 24×24×24 pixels are outputted; and thenentering to a third level. In the third level, after feature extractionusing the 3D 3×3×3 convolution layer with said 128 feature images of24×24×24 pixels, another 128 feature images of 24×24×24 pixels (feature3-1) are obtained; after feature extraction using the 3D 3×3×3convolution layer with a stride of 2, 256 feature images of 12×12×12pixels are outputted; and then entering to a fourth level. In the fourthlevel, after feature extraction using the 2D 3×3×1 convolution layerwith a stride of 2 with said 256 feature images of 12×12×12 pixels(feature 4-1), 512 feature images of 6×6×6 pixels are outputted; andthen a process of feature decoding can be performed.

After the 512 images of 6×6×6 pixels are feature restored by trilinearupsample and added with another 256 feature images of 12×12×12 pixels(which are obtained after the feature 4-1 passes through the 3D 3×3×3convolution layer), 256 feature images of 12×12×12 pixels are outputted.Said 256 feature images of 12×12×12 pixels pass through the 3D 3×3×3convolution layer to generate another 256 feature images of 12×12×12pixels. After said another 256 feature images of 12×12×12 pixels arefeature restored by trilinear upsample and added with another 128feature images of 24×24×24 pixels (which are obtained after the feature3-1 passes through the P3D 3×3×1+1×1×3 convolution layers), 128 featureimages of 24×24×24 pixels are outputted. After said 128 feature imagesof 24×24×24 pixels are feature restored by trilinear upsample and addedwith another 64 feature images of 48×48×48 pixels (which are obtainedafter the feature 2-2 passes through the 3D 3×3×3 convolution layer), 64feature images of 48×48×48 pixels are outputted. Said 64 feature imagesof 48×48×48 pixels (feature 5-1) pass through the 2D 3×3×1 convolutionlayer and another 64 feature images of 48×48×48 pixels (feature 5-2) areobtained. After said another 64 feature images of 48×48×48 pixels passthrough the 3D 3×3×3 convolution layer and add with the feature 5-1,another 64 feature images of 48×48×48 pixels are obtained. After saidanother 64 feature images of 48×48×48 pixels pass through the stem 3×3×3convolution layer, another 64 feature images of 48×48×48 pixels areobtained. After said another 64 feature images of 48×48×48 pixels andthe feature 5-2 are feature restored by trilinear upsample and addedtogether, 32 feature images of 96×96×96 pixels are outputted. Finally,after performing feature decode to said 32 feature images of 96×96×96pixels via the stem 3×3×3 convolution layer, one feature image of96×96×96 pixels is outputted. At this time, the size of the featureimage of the last level is equal to the size of the image of the firstlevel.

The first neural network module 11 can input the plurality of images tothe trained first model so as to obtain a result of determined positionsand ranges of an organ and a tumor of the plurality of images and usethe result as a second training set. The result can be a mask of thepositions and ranges of the organ and the tumor, or the result can be animage of the positions and ranges of the organ and the tumor that havebeen marked, and the present disclosure is not limited as such. Thesecond neural network module 12 can train and obtain the second modelsbased on the second training set. The second neural network module 12 isa deep learning model architecture of DenseNet-121. During a trainingprocess, the second neural network module 12 can use Adam optimizer andcosine annealing learning rate scheduler to adjust a learning rate in arange of 10⁻⁴ to 10⁻⁵, a loss function is set to a binary classificationloss, and a batch size is 16. In an embodiment, the second neuralnetwork module 12 can first evenly divide the plurality of images andthe second training set (take the result is a mask of the positions andranges of the organ and the tumor as an example) into a plurality offolds (subsets), and repeatedly use one of the plurality of folds as avalidation set and the others as a training set in a non-repetitivemanner so as to train and obtain the plurality of second models. Forexample, the second neural network module 12 can first evenly divide theplurality of images and the second training set into five folds numbered1 to 5, and train number 1 (using as a validation set) and numbers 2-5(using as a training set) to obtain a first second model. The secondneural network module 12 can train number 2 (using as a validation set)and numbers 1, 3-5 (using as a training set) to obtain a second secondmodel. The second neural network module 12 can train number 3 (using asa validation set) and numbers 1, 2, 4, 5 (using as a training set) toobtain a third second model. The second neural network module 12 cantrain number 4 (using as a validation set) and numbers 1-3, 5 (using asa training set) to obtain a fourth second model. The second neuralnetwork module 12 can train number 5 (using as a validation set) andnumbers 1-4 (using as a training set) to obtain a fifth second model.That is, the number of the folds is equal to the number of the secondmodels trained. The trained second models can output a prediction valuecorresponding to an input image, and the prediction value can be usedfor classification (predicting having cancer or not having cancer).

In an embodiment, a deep learning model architecture of DenseNet-121 canbe as follows: input an image to a 7×7 convolutional layer with a strideof 2; connect to a 3×3 maximum pooling layer with a stride of 2;sequentially pass through a plurality of dense blocks (e.g., four denseblocks) and a plurality of transition blocks (e.g., three transitionblocks); sequentially enter 7×7 global average pooling layer, denselayer and softmax layer; and finally output a prediction value.

The above described the training stages of the first model and thesecond models. In actual application stage, a program can be written todirectly use the output of the first model as the input of the secondmodels so as to achieve automation effect. For example, python (version3.6.8) can be used to write code. The first model and the second modelscan be implemented using Tensorflow software library or NVIDIA ClaraTrain SDK framework (version 3.0). Therefore, a user only needs to inputone or a plurality of patient images to the first model, and the firstmodel will output a result having determined positions and ranges of anorgan and a tumor of the patient image. Said result can be automaticallyinputted to the plurality of second models so as to output a predictionvalue corresponding to the patient image. In the case of one patientimage and a plurality of second models, different prediction values ofone patient image in different second models will be obtained. In thecase of a plurality of patient images and a plurality of second models,a plurality of different prediction values of each of the plurality ofpatient images in different second models will be obtained.

Whether in the training stage or the application stage, the followingmethod can be used to classify the prediction value. The threshold-valueselection module 15 can determine a threshold value to classify theprediction value as predicting having cancer or not having cancer. Forinstance, after obtaining the plurality of prediction values, thethreshold-value selection module 15 uses a specific threshold value todetermine the plurality of prediction values so as to calculatestatistical indicators (which include sensitivity and specificity, etc.)corresponding to the specific threshold value. Further, a plurality ofsensitivities and specificities calculated according to possible valuesof the plurality of specific threshold values can be plotted as a curve,and the curve can be a receiver operating characteristic (ROC) curve.Then, from the receiver operating characteristic curve, statisticalindicators such as area under receiver operating characteristic (AUC)curve, a plurality of Youden indexes, or a combination of sensitivityand specificity, etc. can be obtained. The plurality of Youden indexesare calculated from the sensitivity and the specificity corresponding toeach point in the curve according to the formula: Youdenindex=Sensitivity+Specificity−1. The combination of sensitivity andspecificity is calculated from the sensitivity and the specificitycorresponding to each point in the curve according to the formula:Combination of Sensitivity and Specificity=Sensitivity×N+Specificity,where N is any number (e.g., 1 or 2, or even a number less than 1). Inthe present disclosure, the threshold values corresponding to themaximum values in the plurality of Youden indexes are used as thresholdvalues, or the threshold values corresponding to the maximum values ofthe combination of sensitivity and specificity are used as thresholdvalues, and the present disclosure is not limited as such. When theprediction value of the image (or patient image) is greater than thethreshold value, the prediction value can be classified as predictinghaving cancer (positive), otherwise it can be classified as predictingnot having cancer (negative).

In an embodiment, taking the aforementioned five second models asexamples, one or a plurality of images (or patient images) arerespectively inputted into five second models to obtain five sets ofpredictions values corresponding to the five second models,respectively. Each set of prediction values can include a predictionvalue or a plurality of prediction values according to the difference ofthe number of images inputted. In the case of inputting a plurality ofimages of a patient into a second model to obtain a plurality ofprediction values, another prediction value representing whether havingcancer can be defined by a ratio of the number of images classified aspredicting having cancer in the plurality of images to the total numberof the plurality of images. If a plurality of prediction values areobtained by inputting a plurality of images corresponding to each of theplurality of patients, whether the plurality of prediction values arepredicting having cancer or not can be determined by the aforementionedprocess of curve plotting and the aforementioned process of determiningthe threshold value representing having cancer or not. Since the fivesecond models may have different results for the same set of images, forexample, a few of the five second models may determine having cancer,the determining module 13 is configured to decide how many second modelsdetermining having cancer are needed before outputting the total resultas having cancer.

The determining module 13 first respectively calculates positivelikelihood ratio for the performance of different model numberspredicting having cancer in the plurality of second models so as todecide and determine whether the final result is a number thresholdvalue (e.g., quantity threshold value) of the plurality of second modelsrepresenting positive (i.e., predicting having cancer). The formula ofpositive likelihood ratio is: “the proportion of X models predictingpositive in the ground truth positive image” divided by “the proportionof X models predicting positive in the ground truth negative image,”where each different X corresponds to a positive likelihood ratio. Asshown in Table 1, taking five second models as an example, when zero(X=0) or one (X=1) or two (X=2) models predicting positive areconsidered, the positive likelihood ratio is 0; when three modelspredicting positive (X=3) are considered, the positive likelihood ratiois 0.93; and when four models predicting positive (X=4) are considered,the positive likelihood ratio is 4.25. The determining module 13 selectsthe least number (e.g., least quantity) in the different model numberspredicting having cancer corresponding to a positive likelihood ratiogreater than 1 as the number threshold value. In an embodiment, thenumber threshold value is four second models; as such, in five secondmodels, when four or more than four prediction values corresponding tothe second models predict having cancer, the determined result outputtedby the determining module 13 is having cancer. When only three secondmodels in the five second models determine having cancer, the determinedresult outputted by the determining module 13 is not having cancer.

TABLE 1 The number of The number of The control models predicting cancerpatients number positive in second predicted predicted Positive models(n = 437) (n = 586) likelihood ratio 5 409 0 Inf ( = (409/437)/(0/586))4 19 6 4.25 ( = (19/437)/(6/586)) 3 9 13 0.93 ( = (9/437)/(6/586)) 2 014 0 ( = (0/437)/(14/586)) 1 0 75 0 ( = (0/437)/(75/586)) 0 0 478 0 ( =(0/437)/(478/586))

In an embodiment, whether in the training stage or in the applicationstage, the image preprocessing module 14 can be used to process thepatient image or the image in the database 16 before inputting the firstmodel or the second models. For example, before inputting the firstmodel, the image can be resampled to the same spacing (1 mm, 1 mm, 1 mm)by using a linear interpolation and a nearest-neighbor interpolation,and then the window width and the window level of the image are set to450 HU and 25 HU (Hounsfield unit), respectively. Finally, the image isnormalized, that is, the pixel intensity value of the image is setbetween 0 and 1. For another example, before inputting the secondmodels, a portion without the organ position and range markers and thetumor position and range markers in the image can first be removed, andthen fragments less than 1000 voxels (about 200 mm³) are removed so asto prevent the second models from generating deviations. Next, the imagecan be resampled to the same spacing (1 mm, 1 mm, 5 mm) by using alinear interpolation and a nearest-neighbor interpolation, and then thewindow width and the window level of the image are set to 250 HU and 75HU, respectively. Finally, the image is normalized, that is, the pixelintensity value of the image is set between 0 and 1.

Referring to FIG. 2, a medical image analyzing method according to thepresent disclosure is further provided. The medical image analyzingmethod according to the present disclosure can be used in the medicalimage analyzing system 1. The same technical content between the medicalimage analyzing method according to the present disclosure and themedical image analyzing system will not be repeated herein.

In step S1, at least one patient image is obtained first. In step S2,the patient image is inputted to a first model of a first neural networkmodule 11 so as to obtain a result having determined positions andranges of an organ and a tumor of the patient image. In step S3, theresult having the determined positions and ranges of the organ and thetumor of the patient image is respectively inputted to a plurality ofsecond models of a second neural network module 12 so as to obtain aplurality of prediction values corresponding to each of the plurality ofsecond models and a model number predicting having cancer in theplurality of prediction values. In step S4, a determining module 13outputs a determined result based on the model number predicting havingcancer and a number threshold value. The training methods of the firstmodel and the second models are the same as the training methods of thefirst model and the second models in the medical image analyzing systemand will not be repeated herein.

In an embodiment, step S3 further includes the following steps: thethreshold-value selection module 15 plots a plurality of curves for theplurality of prediction values, so a plurality of threshold values fordetermining whether there is cancer can be determined from each of theplurality of curves, so that the second neural network module 12 candetermine whether the plurality of prediction values predict havingcancer based on the plurality of threshold values.

In an embodiment, in step S4, the determining module 13 respectivelycalculates the corresponding positive likelihood ratio for theperformance of different model numbers predicting having cancer in theplurality of second models so as to determine the number thresholdvalue; and when the model number predicting having cancer is greaterthan or equal to the number threshold value, a determined result ofhaving cancer is outputted.

In an embodiment, the following step can be performed after step S1 andbefore step S2, or after step S2 and before step S3: the imagepreprocessing module 14 processes the image (or patient image) byresampling, windowing and normalization.

The efficacy of the medical image analyzing system and method accordingto the present disclosure is verified as follows: first, 718 pancreaticcancer patients and 698 computer tomography images of healthy pancreasesare provided; one first model and five second models are generated bytraining; the second models achieve a sensitivity of 89.9% (95%confidence interval, 82.7%-94.9%) and a specificity of 95.9% (95%confidence interval, 91.3%-98.5%), an area under the curve (AUC) is0.964 (95% confidence interval, 0.942-0.986); and when the tumor size isless than 2 cm, the second models achieve a sensitivity of 87.5% (95%confidence interval, 67.6%-97.3%).

In conclusion, the medical image analyzing system and method accordingto the present disclosure have higher sensitivity than radiologists inidentifying pancreatic cancer, which means that the medical imageanalyzing system and method according to the present disclosure caneffectively assist radiologists in reducing their clinical misseddiagnosis rate, especially in the case of tumors less than 2 cm in size.Therefore, the situation that about 40% of the tumors cannot be detectedwhen the tumor is less than 2 cm in size in the general clinicalsituation can be effectively improved. In addition, the medical imageanalyzing system and method according to the present disclosure areautomated processes. After directly inputting the original medicalimage, the medical image analyzing system and method according to thepresent disclosure can automatically identify the possible positions andranges of pancreas and tumor and automatically classify whether theoriginal medical image contains pancreatic cancer, so that the presentdisclosure can be easy to use.

The above-described descriptions of the detailed embodiments are toillustrate the implementation according to the present disclosure, andit is not to limit the scope of the present disclosure. Accordingly, allmodifications and variations completed by those with ordinary skill inthe art should fall within the scope of present disclosure defined bythe appended claims.

What is claimed is:
 1. A medical image analyzing system, comprising: afirst neural network module having a first model and configured to inputat least one patient image into the first model to obtain a resulthaving determined positions and ranges of an organ and a tumor of thepatient image; a second neural network module having a plurality ofsecond models and configured to respectively input the result having thedetermined positions and ranges of the organ and the tumor of thepatient image into the plurality of second models to obtain a pluralityof prediction values corresponding to each of the plurality of secondmodels and a model number predicting having cancer in the plurality ofprediction values; and a determining module configured to output adetermined result based on the model number predicting having cancer anda number threshold value.
 2. The medical image analyzing system of claim1, further comprising a database stored with a plurality of images,organ position and range markers and tumor position and range markers,wherein the plurality of images, the organ position and range markersand the tumor position and range markers are interlinked to use as afirst training set.
 3. The medical image analyzing system of claim 2,wherein the first neural network module is trained to obtain the firstmodel based on the first training set, wherein the first neural networkmodule is a model searched by using a coarse-to-fine neural structuresearch (C2FNAS), and wherein the first neural network module uses anAdam optimizer and a cosine annealing learning rate scheduler to adjusta learning rate in a range of 10⁻⁴ to 10⁻⁵, and a loss function is setto a Dice loss combined with a categorical cross-entropy loss.
 4. Themedical image analyzing system of claim 3, wherein the first neuralnetwork module obtains a result having determined positions and rangesof an organ and a tumor of the plurality of images by inputting theplurality of images into the first model and uses the result as a secondtraining set.
 5. The medical image analyzing system of claim 4, whereinthe second neural network module is trained to obtain the plurality ofsecond models based on the second training set, wherein the secondneural network module is a DenseNet-121, and wherein the second neuralnetwork module uses an Adam optimizer and a cosine annealing learningrate scheduler to adjust a learning rate in a range of 10⁻⁴ to 10⁻⁵, anda loss function is set to a binary classification loss.
 6. The medicalimage analyzing system of claim 5, wherein the second neural networkmodule evenly divides the plurality of images and the second trainingset into a plurality of folds, and repeatedly uses one of the pluralityof folds as a validation set and the others of the plurality of folds asa training set in a non-repetitive manner to train and obtain theplurality of second models.
 7. The medical image analyzing system ofclaim 1, further comprising a threshold-value selection moduleconfigured to plot a plurality of curves for the plurality of predictionvalues, wherein a plurality of threshold values for determining whetherthere is cancer are determined from each of the plurality of curves,such that the second neural network module determines whether theplurality of prediction values predict having cancer based on theplurality of threshold values.
 8. The medical image analyzing system ofclaim 7, wherein the plurality of curves are receiver operatingcharacteristic curves, and wherein the plurality of threshold values arecorresponding threshold values corresponding to maximum values of aplurality of Youden indexes or a combination of a sensitivity and aspecificity.
 9. The medical image analyzing system of claim 8, whereinthe plurality of Youden indexes are calculated from the sensitivity andthe specificity corresponding to each point in each of the plurality ofcurves according to a formula Youden index=Sensitivity+Specificity−1,wherein the combination of the sensitivity and the specificity arecalculated from the sensitivity and the specificity corresponding to theeach point in each of the plurality of curves according to a formulaCombination of Sensitivity and Specificity=Sensitivity×N+Specificity,and wherein the N is any number.
 10. The medical image analyzing systemof claim 1, wherein the determining module respectively calculatescorresponding positive likelihood ratios for performance of differentmodel numbers predicting having cancer in the plurality of second modelsto determine the number threshold value, and wherein the determiningmodule outputs the determined result of having cancer when the modelnumber predicting having cancer is greater than or equal to the numberthreshold value.
 11. The medical image analyzing system of claim 10,wherein the determining module selects a least number in the differentmodel numbers predicting having cancer corresponding to the positivelikelihood ratio greater than 1 as the number threshold value.
 12. Themedical image analyzing system of claim 1, further comprising an imagepreprocessing module configured to process the patient image byresampling, windowing and normalization before inputting the first modelor the plurality of second models.
 13. A medical image analyzing method,comprising: obtaining at least one patient image; inputting the patientimage into a first model of a first neural network module to obtain aresult having determined positions and ranges of an organ and a tumor ofthe patient image; inputting the result having the determined positionsand ranges of the organ and the tumor of the patient image into theplurality of second models of a second neural network module,respectively, to obtain a plurality of prediction values correspondingto each of the plurality of second models and a model number predictinghaving cancer in the plurality of prediction values; and outputting adetermined result by a determining module according to the model numberpredicting having cancer and a number threshold value.
 14. The medicalimage analyzing method of claim 13, further comprising interlinking aplurality of images, organ position and range markers and tumor positionand range markers to use as a first training set via a database storedwith the plurality of images, the organ position and range markers andthe tumor position and range markers.
 15. The medical image analyzingmethod of claim 14, wherein the first neural network module is trainedto obtain the first model based on the first training set, wherein thefirst neural network module is a model searched by using acoarse-to-fine neural structure search (C2FNAS), and wherein the firstneural network module uses an Adam optimizer and a cosine annealinglearning rate scheduler to adjust a learning rate in a range of 10⁻⁴ to10⁻⁵, and a loss function is set to a Dice loss combined with acategorical cross-entropy loss.
 16. The medical image analyzing methodof claim 15, wherein the first neural network module obtains a resulthaving determined positions and ranges of an organ and a tumor of theplurality of images by inputting the plurality of images into the firstmodel and uses the result as a second training set.
 17. The medicalimage analyzing method of claim 16, wherein the second neural networkmodule is trained to obtain the plurality of second models based on thesecond training set, wherein the second neural network module is aDenseNet-121, and wherein the second neural network module uses an Adamoptimizer and a cosine annealing learning rate scheduler to adjust alearning rate in a range of 10⁻⁴ to 10⁻⁵, and a loss function is set toa binary classification loss.
 18. The medical image analyzing method ofclaim 17, wherein the second neural network module evenly divides theplurality of images and the second training set into a plurality offolds, and repeatedly uses one of the plurality of folds as a validationset and the others of the plurality of folds as a training set in anon-repetitive manner to train and obtain the plurality of secondmodels.
 19. The medical image analyzing method of claim 13, furthercomprising plotting a plurality of curves for the plurality ofprediction values by a threshold-value selection module, wherein aplurality of threshold values for determining whether there is cancerare determined from each of the plurality of curves, such that thesecond neural network module determines whether the plurality ofprediction values predict having cancer based on the plurality ofthreshold values.
 20. The medical image analyzing method of claim 19,wherein the plurality of curves are receiver operating characteristiccurves, and wherein the plurality of threshold values are correspondingthreshold values corresponding to maximum values of a plurality ofYouden indexes or a combination of a sensitivity and a specificity. 21.The medical image analyzing method of claim 20, wherein the plurality ofYouden indexes are calculated from the sensitivity and the specificitycorresponding to each point in each of the plurality of curves accordingto a formula Youden index=Sensitivity+Specificity−1, wherein thecombination of the sensitivity and the specificity are calculated fromthe sensitivity and the specificity corresponding to the each point ineach of the plurality of curves according to a formula Combination ofSensitivity and Specificity=Sensitivity×N+Specificity, and wherein the Nis any number.
 22. The medical image analyzing method of claim 13,wherein the determining module respectively calculates correspondingpositive likelihood ratios for performance of different model numberspredicting having cancer in the plurality of second models to determinethe number threshold value, and wherein the determining module outputsthe determined result of having cancer when the model number predictinghaving cancer is greater than or equal to the number threshold value.23. The medical image analyzing method of claim 22, wherein thedetermining module selects a least number in the different model numberspredicting having cancer corresponding to the positive likelihood ratiogreater than 1 as the number threshold value.
 24. The medical imageanalyzing method of claim 13, further comprising enabling an imagepreprocessing module to process the patient image by resampling,windowing and normalization before inputting the first model or theplurality of the second models.